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Subject: Homeland Security Grant Program Risk-Based Distribution Methods: 
Presentation to Congressional Committees - November 14, 2008 and December 15, 
2008 

This report formally transmits the attached briefing in response to P.L. 110-329, the 
Consolidated Security, Disaster Assistance and Continuing Appropriations Act, that 
required GAO for the fourth year to review the methodology the Department of 
Homeland Security (DHS) and the Federal Emergency Management Agency (FEMA) use 
to allocate Homeland Security Grant Program (HSGP) grants, including the risk 
assessment methodology they use to determine which urban areas are eligible to apply 
for grants. HSGP includes the State Homeland Security Program (SHSP) and Urban 
Areas Security Initiative (UASI) grants. Our objective was to identify any changes in the 
methodology for risk assessment and grant allocation for 2009 and to assess the 
reasonableness of the methodology. We analyzed DHS and FEMA documents including 
the fiscal year 2008 and 2009 risk analysis models and grant guidance and interviewed 
DHS and FEMA officials about the changes in the 2009 model. We did our work between 
October and December 2008 in accordance with generally accepted government auditing 
standards. We briefed the mandated reporting committees with two briefings in 
November 2008 and December 2008 on the results of our analysis. A copy of the final 
briefing is enclosed. 

DHS has adopted a process of continuing improvement in its methods for assessing risk 
and measuring grant applicants' effective use of resources. SHSP and UASI grant 
allocations continue to be based on a three-step process: (1) risk assessments to 
determine areas eligible to apply for grants, (2) effectiveness assessments of the grant 
applicants' investment justifications, and (3) final grant allocations. The methodology is 
described in detail in our 2008 report. 1 There were minor changes in the risk assessment 
for 2009 and no changes in the process used for the effectiveness assessment and final 
grant allocations. Generally, we found that DHS has constructed a reasonable 
methodology to assess risk and allocate funds. DHS uses empirical risk analysis and 
policy judgments to select the urban areas eligible for grants (all SHSP grantees are 



1 GAO, Homeland Security: DHS Risk-Based Methodology is Reasonable, But Current Version's Measure of 
Vulnerability is Limited, GAO-08-852 (Washington, D.C.: June 2008). 
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guaranteed a specified minimum percentage of available grant funds) and to allocate 
SHSP and UASI funds. 

DHS continued to include three basic variables in its risk assessment — threat, 
vulnerability, and consequences. For fiscal year 2009, DHS changed some elements of 
the inputs used for the threat, economic, and national security indexes used in the 
model. The value of each of these indices is given a specific weight in the risk 
assessment (for example, the threat index is weighted at 20 percent and the population 
index at 40 percent of the final risk score). DHS continued to consider all areas of the 
nation equally vulnerable to a successful terrorist attack and assigned every state and 
urban area a vulnerability score of 1.0 in the risk analysis model. Thus, as a practical 
matter, the final risk scores are determined by the threat and consequences scores. 

The result of the UASI risk assessment is a list of UASI jurisdictions eligible for grants. A 
total of 62 urban areas were eligible for grants in 2009 (60 were eligible in 2008). In 
response to grantee feedback, in 2009, DHS for the first time provided each eligible State 
and UASI area its estimated target grant allocation in the Fiscal Year 2009 Homeland 
Security Grant Program Guidance and Application Kit. FEMA believes this change will 
enable grantees to better target their investment justifications in their grant applications 
as the amounts requested should more closely match final actual allocations. Applicants 
may submit applications for up to 110 percent of their targeted allocation. Final 2009 
allocations will be based on the targeted allocations as adjusted by applicants' 
effectiveness scores, up to a maximum of plus or minus 10 percent of their targeted 
allocation. As in 2008, the seven urban areas with the highest risk scores (Tier I) will be 
collectively allocated 55 percent of total available fiscal year 2009 funds and the 
remaining 55 urban areas (Tier II) will be collectively allocated 45 percent. 

We are not making any new recommendations for congressional consideration or agency 
action. However, we note that DHS has not taken action to address our recommendation 
from last year's report that "the Secretary of DHS formulate a method to measure 
vulnerability that captures variations across states and urban areas, and apply this 
vulnerability measure in future iterations of this risk-based grant allocation model." 
FEMA and DHS concurred with the recommendation. In its comments on a draft of our 
briefing slides, FEMA officials stated that their ability to make a substantial change to 
the vulnerability calculation was limited by (1) the timing of the fiscal year 2009 roll-out 
"on the heels" of the fiscal year 2008 allocations and (2) DHS leadership's desire to 
minimize year-to-year changes. We continue to believe that FEMA should formulate a 
method to measure vulnerability that captures variations across jurisdictions. DHS has 
completed vulnerability assessments for critical infrastructure and surface 
transportation modes that could be of potential use in assessing variations in 
vulnerability across jurisdictions. 

FEMA's officials also told us that the Grant Directorate is developing a Cost-to-Capability 
(C2C) initiative that could, among its other purposes, potentially serve as a proxy 
measure for vulnerability to an international terrorist incident. Still in its early stages, 
C2C focuses on efforts to measure a jurisdiction's capability to prevent and respond to 
various types of disasters compared to a target level of capability. To measure 
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capability, the C2C initiative will use DHS's 15 National Planning Scenarios of terrorist 
and nonterrorist disaster incidents, and the target capabilities developed using those 
scenarios. Those target capabilities are currently being revised. With regard to 
vulnerability, the concept behind C2C is that the more capable an area is to prevent and 
respond to a terrorist attack, the less vulnerable it is. However, as designed, C2C results 
will not directly measure vulnerability. In its agency comments FEMA noted that the 
extent to which C2C will help measure vulnerability is yet to be determined. 



We are sending copies of this report to the appropriate congressional committees, the 
Secretary of Homeland Security, and other interested parties. This report will also be 
available at no charge on our Web site at http://www.gao.gov. Should you or your staff 
have any questions concerning this report, please contact me at (202) 512-8757 or 
jenkinsWO@gao.gov. Contact points for our Offices of Congressional Relations and 
Public Affairs may be found on the last page of this report. Key contributors to this 
report were Chris Keisling, Assistant Director; Charles Bausell, Assistant Director; John 
Vocino; Orlando Copeland; and Perry Lusk. 




William O. Jenkins 

Director, Homeland Security and Justice Issues 
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List of Committees 



The Honorable Robert C. Byrd, Chairman 
The Honorable Thad Cochran, Ranking Member 
Committee on Appropriations 
United States Senate 



The Honorable David R. Obey, Chair 

The Honorable Jerry Lewis, Ranking Member 

Committee on Appropriations 

House of Representatives 
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Introduction 



According to the Department of Homeland Security (DHS): 

• DHS will make available approximately $1 .7 billion to states and 
urban areas through its FY 2009 Homeland Security Grant Program 
(HSGP) to prevent, protect against, respond to, and recover from 
acts of terrorism or other catastrophic events. 

• The HSGP risk-based allocation process is used for the State 
Homeland Security Program (SHSP) and Urban Areas Security 
Initiative (UASI). 

• DHS's grant programs are managed by the Federal Emergency 
Management Agency's (FEMA) Grants Program Directorate. 



Page 6 



GAO-09-168R Homeland Security Grant Risk Assessment 



Enclosure: Briefing Slides 



j- G AP 

mfl^^^ Accountability * Integrity * Reliability 



Objectives 



Since FY 2006, GAO has responded to a legislative mandate to 
review the grant program and assess its allocation method. 

This year we are addressing the following question: 



What methodology did DHS use to allocate HSGP 
funds, including any changes DHS made to the eligibility 
and allocation processes, and how reasonable is DHS's 
methodology? 



Page 7 



GAO-09-168R Homeland Security Grant Risk Assessment 



Enclosure: Briefing Slides 



^. GAP 

nH^^™ Accountability * Integrity * Reliability 



Scope and Methodology 



We analyzed DHS documents including the FY 2008 and FY 2009 risk 
analysis models and grant guidance, and interviewed DHS officials about: 

• The changes, if any, to the HSGP FY 2009 grant eligibility and 
allocation processes: 

• The process by which DHS's risk analysis model is used to 
estimate relative risk: Risk= Threat x "Vulnerability & 
Consequences." 

• How the effectiveness assessment process is conducted. 

• How final allocation decisions are made. 

We did our work from October 2008 to December 2008 in accordance with 
generally accepted government auditing standards. 

We initially briefed congressional staff on November 14, 2008, adding 

information on the Cost-to-Capability initiative for the December 15 briefing. 



Page 8 



GAO-09-168R Homeland Security Grant Risk Assessment 



Enclosure: Briefing Slides 



j- G AP 

mfl^^^ Accountability * Integrity * Reliability 



Results in Brief 



Our 2009 review found that compared to 2008: 

• There was no change in DHS's basic approach. 

• Allocation method remains reasonable, but Vulnerability measure remains limited. 

• DHS used same three-step process: (1) risk analysis to determine eligible UASI 
areas, (2) effectiveness assessment of applicants' investment proposals, and (3) 
final allocation decisions. 



• Risk analysis model = minimal changes. 

• Criteria for one Threat Index tier changed. 

• Data source within Economic Index changed; two sources added in 
National Security Index. 

• Risk analysis results in list of UASI areas eligible for funding. New in FY 
2009, DHS orovidedarantees their estimated target allocations in the 
Homeland Security Grant Program Guidance ana Application Kit. 



• Effectiveness assessment process = no changes. 



• Final grant allocation = no changes 



5 
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Background 



We've reviewed this program for the last four grant cycles. In previous 
reviews we reported: 

• Inherent uncertainty is associated with estimating risk of terrorist 
attack, requiring the application of policy and analytic judgments. 
The use of sensitivity analysis can help to gauge what effects key 
sources of uncertainty have on outcomes. 

• DHS has adopted a process of "continuing improvement" to its 
methods for estimating risk and measuring applicants' 
effectiveness. 
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Background: Evolution of DHS's Risk-Based 
Formula 



Department of 
Justice - run 
grant program 



10/26/01 

USA PATRIOT Act 



11/25/02 

Homeland Security Act 



STAGE I 
R=P 




FYs 2001-2003 



FYs 2004-2005 



STAGE III 




R=T» V *C 




FY 2006 


FYs 2007-2009 



Source: GAO analysis based on Congressional Research Service. 



Definitions for the formulas above: 

• R = P represents Risk = Population 

• R = T+CI+PD represents Risk = Threat plus Critical Infrastructure plus Population Density 

• R = T*V*C represents Risk = Threat times Vulnerability times Consequences 

• R = T* "V&C" represents DHS's presentation of the risk calculation formula used in their risk analysis model for 2007-2009. 
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Background: Overview of the HGSP Grant 
Determination Process (UASI and SHSP) 

Since 2006, DHS has applied a three-step process — using analytical 
methods and policy judgments — to select eligible urban areas and 
allocate SHSP and UASI funds: 

1 . Use of a risk analysis formula (R = T times " V&C) with the 
same basic indices (e.g., threat, economic), and weights 
results in list of eligible UASI areas. 

2. Implementation of an effectiveness assessment, including a 
peer review process, to assess and score the effectiveness of 
the proposed investments submitted by the eligible applicants. 

3. Calculation of a final allocation of funds based on state's 
and urban areas' risk scores as adjusted by their 
effectiveness scores. 
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Background: Overview of the Grant Allocation 
Methodology for UASI and SHSG 



Phase 1 : Risk analysis 



Funding 
allocation 



Relative risk 
estimate 



Tier 1 1 Tier 2 



Cutoff 
point 




Phase 2: Effectiveness assessment HPhase 3: Final allocation 



Effectiveness/risk matrix 



1 10 20 30 40 50 
Number of urban areas 



70 80 90 100 




J $ 

$ * $ ^ 



Risk estimator 
R = T x "V&C" 



Peer review 
of Investment 
Justifications 



Relative risk 
estimate 





Effectiveness score-*- 



Effectiveness/risk matrix 



5 10 15 20 25 30 35 40 45 (56) 
Number of states and territories 



$ $ 

.$ $ S» 



Effectiveness score-K- 
Statutory minimum = ,365% a 



Source: GAO analysis of DHS documents and information provided in interviews. 



' Statutory minimum for FY 2009 = 0.365% of the total amount of funds appropriated for SHSP plus UASI grants. 
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Risk Analysis in FY 2009: Model Used in 
Determining Relative Risk Scores 



Data: Ongoing plotlines, credible 
reporting, relevant investigations to 
create threat tiers 

Source: Intelligence Community 
reporting 




Vulnerability and Consequence Index 
V&C = (P+E+l+N) 



Data: Population (Nighttime, 
Commuter, Visitor) and Populations 
Density 

Source: Census, Smith Travel 



Economic Index 
Data: Gross Domestic Product 



Source: Department of Commerce, 
Bureau ot Economic Statistics 



National Infrastructure Index 
Data: # Tier 1 Assets (x3) +# Tier II 



Source: DHS/OIR SSAs, states and 
territories. 



National Security Index 

Data: Number of Defense Industrial Base 
(DIB) Facilities, Military Personnel, 
International Border Crossings; Presence 
of Coastline, International Border 

Source: DOD, DHS/CBP 



Note: "DHS/OIP" stands for DHS's Office of Infrastructure Protection. "SSAs" stands for Sector-Specific Agencies, which are federal departments 
and agencies identified in the National Infrastructure Protection Plan as responsible for critical infrastructure protection activities. "DOD" stands 
for the Department of Defense. "DHS/CBP" stands for the DHS's Customs and Border Protection. 
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Risk Analysis Model: Calculating Threat 



Threat Index: 

• Reflects the intelligence community's best assessment of areas 
of the country and potential targets most likely to be attacked. 

• Process considers threat information from the intelligence 
community of ongoing plotlines and credible threats from 
international terrorist networks and affiliates. 



For FY 2009 model, criteria for one of the four threat tiers changed to 
reflect updated intelligence assessment. 

DHS — Intelligence and Analysis (l&A) — reported additional processes 
implemented to consult with states on threat information. 
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Risk Analysis Model: Calculating Vulnerability 
and Consequences ("1/&C") 

• Few changes made to the model, primarily data and process 
improvements, according to DHS officials. 

• Vulnerability is still considered to be constant across all jurisdictions and 
in the risk formula is assigned a value of 1 .0 for all jurisdictions. 

Consequences: 

Population Index: 

• No changes. 

Economic Index: 

• Data source changed from that of a private company to an annually 
updated data set from the U.S. Commerce Department's Bureau of 
Economic Analysis. 
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Risk Analysis Model: Calculating Vulnerability 
and Consequences (" V&C) (continued) 

National Infrastructure Index: 

• No changes. 

National Security Index: 

• Additional data sources were added to improve the index. 

1 . The number of military personnel and the number of critical 
defense industrial base facilities. 

2. Presence of borders or coastline as determined by U.S. 
Customs and Border Protection inspection. 

• Each jurisdiction received either full credit or none. 
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Risk Analysis Model: Calculating Vulnerability 
and Consequences (" V&C) (continued) 

Vulnerability is a crucial component of risk assessment. 
In June 2008, we reported that 1 

• While DHS has constructed a reasonable methodology to 
allocate funds within a given year, the Department needs to 
measure Vulnerability to capture variations in vulnerability 
across states and urban areas. 



As a result, we recommended 

• "that the Secretary of DHS formulate a method to measure 
Vulnerability that captures variations across states and urban 
areas, and apply this Vulnerability measure in future iterations 
of this risk-based grant allocation model." 



1 GAO, Homeland Security: DHS Risk-Based Grant Methodology Is Reasonable, But Current Version's Measure of Vulnerability is 

Limited, GAO-08-852 (Washington, D.C.: June 2008). -\ 4 
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Vulnerability Element of DHS's Risk Analysis 
Model Has Limitations that Reduce Its Value 

Status of DHS's response to our recommendation: 

• FEMA and l&A agreed that the current methodology for determining 
Vulnerability is not the optimum approach. 

• FEMA officials told us they have not identified a means of measuring 
Vulnerability effectively across all states and urban areas, but had little time 
between the 2008 and 2009 cycles to consider alternatives. 

• Instead, FEMA's Grant Directorate has focused its resources on launching 
an inverse approach called Cost-to-Capability (C2C) that is in its early 
stages. C2C focuses on efforts to measure a jurisdiction's capability to 
prevent and respond to various types of disasters compared to a target 
level of capability. 

• As its organizing structure, the C2C initiative uses DHS's National Planning 
Scenarios of terrorist and nonterrorist disaster incidents, and the target 
capabilities developed using those scenarios. 
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Grant Programs Directorate's Cost to 
Capability Initiative (C2C) Being Developed 

• C2C is being developed by FEMA's Grant Management Directorate and is 
intended to be a decision support system to enable that Directorate to 
effectively manage FEMA's grant portfolio and to assist grant recipients 
optimize the use of their grant funds. 

• One critical element of C2C — identifying the importance of each capability 
to each of the 15 planning scenarios used to develop target capabilities — is 
still under development. 

• FEMA's intends to provide C2C as a tool that uses a common language 
and analytical framework to help state and local stakeholders make better 
investment decisions. 

• Grantee use of C2C will not be mandatory. 

• As designed, C2C results will not directly measure Vulnerability or 
preparedness. 
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Cost to Capability Initiative (C2C) Faces 
Implementation Challenges 

• To date, national capability assessments have been based on state self- 
assessments whose comparability, reliability, accuracy, and data validity 
are uncertain. 

• Input data for C2C is to be based on self assessments of capabilities from 
state preparedness plans, estimates of baseline capability, and the 
estimated relative capability improvement expected from a requested level 
of investment. 

• For states to use C2C effectively, the state and local data used for 
assessing state and local capabilities must be in the common language of 
Target Capabilities and must have metrics that are compatible with C2C. 
These metrics are being developed by FEMA's National Preparedness 
Directorate. 

• The potential for successful use of C2C as a national capability 
assessment tool is limited by the variations in the analytical skill levels 
across state and local users of C2C, according to FEMA. 
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FEMA/ Grant Programs Directorate's Cost to 
Capability Initiative (C2C) 



Step 1: 

Connect Grant Cellars 
Awantwl Id Planned Activities 



I 1 

Analysis of Investment Purpose 

Gram Applications, Progress 
Reporting, Monitoring Fteiorts, and , 
Grant Closeout Reports 



Analysis of Investment Cost 

Award Notices, Progress 
Reporting. Grant Closeout Reports 



Step 2: 

Use a Capability Framework to Align 
Grant Man « Common Variables 
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DHS Has Not Used Data that Could Inform Risk 
Model Vulnerability Inputs 

DHS has made some efforts to measure vulnerability for specific assets 
across the nation. We reported in 2007 that DHS had^ 

• Assessed vulnerability of assets within surface transportation modes 
(mass transit, freight rail, and highway infrastructure). 

• Conducted over 2,600 vulnerability assessments on every critical 
infrastructure sector through the Comprehensive Review program, 
the Buffer Zone Protection program, and the Site Assistance visit 
program. 2 

DHS and FEMA could evaluate how these assessments could be used to 
provide some differentiation in measures of Vulnerability used for its risk 
model formula. 

1 GAO, Department Of Homeland Security: Progress Report on Implementation of Mission and Management Functions, GAO-07-454 
(Washington, D.C.: August 2007). 

2 DHS describes the Comprehensive Review as a structured, collaborative government and private sector analysis of high value critical 
infrastructure and key resources facilities. Through the Buffer Zone Protection Program, and with the support of DHS, local authorities develop 
Buffer Zone Protection Plans, which DHS reported have several purposes, including identifying specific threats and vulnerabilities associated 
with the buffer zone and analyzing the level of risk associated with each vulnerability. DHS describes the Site Assistance Visit Program as an 
information gathering visit with several goals, such as better understanding and prioritizing vulnerabilities of critical infrastructure and key 
resources and increasing awareness of threats and vulnerabilities among critical infrastructure and key resources owners and operators. 
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Sensitivity of the Risk Analysis Model 



GAO's analysis of the FY 2009 model: 



• It takes moderate changes to the weights of the model's risk 
indices, such as the weights given to the threat or economic 
indices, to change the urban areas that compose the Tier 1 list, 
which are those areas with the highest risk scores. 

• The current model is more sensitive than the FY 2008 model. 



• For those urban areas ranked near the bottom of the Tier 2 list, 
modest changes in the weights for the indices used to quantify risk 
can result in changes in eligibility. 1 

• The current model is more sensitive than the FY 2008 model. 
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Effectiveness Assessment 



No change in process for FY 2009. 
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Final Allocation Process - FY 2009 Grants 
Based on Both Risk and Effectiveness Scores 

DHS/FEMA plans to allocate FY 2009 funds based on the risk scores of states 
and urban areas, adjusted by a maximum of +/-10% by their effectiveness 
scores. 

2009 HSGP Grant Guidance provided grantees with their estimated target 
grant allocations, rather than get allocations after effectiveness assessment: 

• FEMA responded to grantees' input in the 2008 grant after-action 
conference. 

• FEMA believes it enables grantees to better target the grant 
justifications in their applications as amounts requested should more 
closely match final actual allocations. 

• FEMA officials anticipate this change will encourage grantees to 
develop more targeted and realistic proposals that will be better linked 
to capabilities and performance measures. 
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Final Allocation Process - Ranking UASI 
Grantees by Tiered Groups 

62 eligible UASI areas in FY 2009 (60 in 2008): 

• Tier 1 = 7 highest risk areas and eligible for 55% of available funds. 

• Tier 2 = 55 areas (2 more than FY 2008) and eligible for 45% of 
available funds. 

• Same allocation between Tiers 1 and 2 as 2008. 
According to DHS officials: 

• Expansion from 60 to 62 eligible UASI areas for FY 2009 is a policy 
decision driven by desire to provide program consistency for 
grantees. 

• Department intends to review all Urban Areas for next grant cycle -- 
with the new DHS leadership -- to determine the appropriate 
number for the UASI program. 
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Concluding Observations 



The FY 2009 HSGP risk allocation method continues the same approach we 
deemed reasonable in 2008 and provides program continuity. 

FEMA's continuing improvement efforts are reflected in changes including 
data sets used, and providing grantees their target allocations at the 
beginning of the FY 2009 grant cycle. Previously, grantees did not receive 
target allocations, but only their final allocations following the completion of 
the effectiveness assessment. 

GAO's 2008 recommendation on Vulnerability measure remains 
unaddressed. 

• FEMA's C2C initiative does not address our concerns about 
measuring Vulnerability. 

GAO is not issuing new recommendations with this report. 
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Agency Comments 



FEMA provided comments on both the November 14 and December 15 
versions drafts of this briefing. With respect to the Vulnerability element of 
DHS's Risk Analysis Model, FEMA provided written comments via e-mail 
that: 

• It believes its ability to make a substantial change to the Vulnerability 
calculation was limited by (1) the timing of its FY 2009 roll-out "on the 
heels" of FY 2008 and (2) the department's leadership desire to minimize 
year-to-year changes. 

• "it has been suggested that DHS should identify a distinct Vulnerability term 
for each of the four risk indices and multiply it against the Consequence 
terms, which would result in a more precise risk allocation that reflects 
Vulnerability as well as Threat and Consequence." 

FEMA also provided technical comments that we incorporated as appropriate. 
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